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In this article, we use the findings of a study conducted in a university in the 
southeast of Mexico to examine strengths and limitations of portfolios to assess 
formatively the quality of teaching. The research is part of the study: Model for the 
Development and Evaluation of Academic Competencies, involving researchers 
from six Mexican universities. Findings of the study indicate that the use of 
portfolio allowed faculty members to collect evidence of the quality of their work, 
analyze their classroom instruction and began to involve in a dialogue for 
improving teaching. 
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En este artlculo, usamos los resultados de un estudio que se lleva a cabo en una 
Universidad del sureste de Mexico para examinar las fortalezas y limitaciones de 
los portafolios para evaluar de manera formativa la calidad de la docencia. La 
investigacion es parte del estudio: Modelo para el Desarrollo y Evaluacion de 
Competencias Academicas e involucra a investigadores de seis universidades 
Mexicanas. Los resultados de la investigacion indican que el uso del portafolio 
permitio a los docentes colectar evidencia sobre la calidad de su trabajo, analizar su 
practica en el aula y comenzar a involucrarse en un dialogo para mejorar la 
docencia. 
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1. Introduction 1 

Among the three main functions of the university, the one that presents more challenges 
for evaluating is teaching. This happens because of its complexity, since the instructor is 
only one among several factors that can influence student learning. As Ernest Pascarella 
( 2006 ) and Mary Ann Clark et al. ( 2007 ) state, the intellectual and personal development 
of students are affected by countless influences. Some of them derive from the individual 
student and his or her family 2 , curriculum content, classroom environment, goals and 
institutional models, as well as the way in which these are understood and interpreted 
by administrators and teachers. 

In addition, student learning is influenced by the organizational climate, the 
environment external to the school, which includes the sociocultural context, norms and 
a vision about the kind of life story we are partly to prepare. Student cognitive and 
personal development are also much influenced by the influence of peers and other 
groups. 

Regarding the instructors, there is also a variety of factors influencing their capacity to 
influence student learning. Among them, there is instructor proficiency in the course 
content, the didactics of the content knowledge, the instructor’s capacity to adapt his or 
her teaching to the characteristics and needs of students, the teaching context, the way 
in which the content is taught and the way in which it contributes to the 
accomplishment of the learning objectives of other curriculum courses and to student 
development, and more (Stake & Cisneros-Cohernour, 2004 ). 

In spite of the complexity and multiplicity of factors, most teaching evaluations are 
based on student ratings of teachers, collected by a questionnaire that is provided to 
students at the end of the course. It is presumed that, although crude, there will be 
substantial correlation between these ratings and all the other data that could be 
collected. This view is common around the world, and no less so in Latin America 
(Arbesu & Argumedo, 2010 ; Rueda, 2009 ). 

Student perceptions of teaching are important, but too insensitive to the abiding 
complexity. The decision to use a single source of information and disregard the 
complexities of teaching raise issues about the validity of the evaluation results and its 
interpretation. As Stake (2000) asserts, the quality of teaching should be judged in the 
context in which it takes place, considering more than a few influences. Otherwise, 
decisions from the evaluation will have low validity. 

Conducting evaluation studies about faculty development in response to teaching 
complexities is consistent with research conducted in the field of faculty evaluation 
during the last forty years. In addition, it is essential to take into consideration that 
most of the questionnaires used for evaluating teaching are not only a limited source of 
information, they have not been validated in across contexts and cultures 3 . For example, 


1 Financiamiento: El estudio de investigation en el que se basa este artlculo fue financiado por el Programa 
de Mejoramiento del Profesorado (PROMEP) de la Secretarfa de Educacion Publica, Mexico. 

2 According to the literature, among the factors related to student learning at the individual level are: 
perseverance and determination, self-confidence and own capacities, family influences and friends (Clark et 
al., 2007). 

3 As John Ory and Kathryn Ryan (2001) state, much more research is needed on the external validity of 
student perceptions of college teaching and the factors influencing the ratings. 


210 



Revista Iberoamericana de Evaluacion Educativa 


in 1999, Daniel Pratt et al. conducted a study of teaching evaluation in Hong-Kong and 
found that the questionnaires used for evaluating teaching, based on the original 
versions developed in the US, Great Britain, Australia and Canada, were of little use for 
assessing the quality of teaching in Eastern nations. As these authors state: “ the 
practice of using questionnaires and scales developed in other countries, used to obtain 
information about the quality of teaching in Hong-Kong and China can result in the 
imposition of values from another culture, particularly in relation to the appropriate 
role, responsibilities and the form in which professors relate to their students” (p. l). 

In response to the need for additional forms of evaluation, during the 90’s, began the use 
and popularity of portfolios as instruments for assessing teaching. A portfolio is a 
collection of products and artifacts that demonstrates the merit, range, history, 
situationality and complexity of a person’s work that provides evidence of his or her 
activities and accomplishments. Portfolios can be used for assessing both students and 
teachers (Wolf, 1996). 

In this work, we use the findings of a study conducted in a university in the southeast of 
Mexico to examine strengths and limitations of portfolios to assess formatively the 
quality of teaching. The research is part of the study: Model for the Development and 
Evaluation of Academic Competencies, involving researchers from six universities: 
Universidad Autonoma de Chihuahua, la Universidad de Baja California Campus 
Ensenada, la Universidad Autonoma de Yucatan, as! como de la Universidad Nacional 
Autonoma de Mexico, Universidad Iberoamericana y Universidad Autonoma 
Metropolitana. 


2. Method 

Consistent with the methodology of the Model for the Development and Evaluation of 
Academic Competencies, college instructors from the College of Business 
Administration of a Southern Mexican university, were invited to participate in a 
workshop. The instructors developed their own teaching portfolios and analyzed the 
conditions of teaching and its evaluation at their university. In addition, the instructors 
analyzed the utility of the portfolio as an instrument for personal reflection and its 
pertinence for assessing the quality of their teaching. A portfolio was to focus partly on 
the following characteristics of the instructor’s teaching practice: 


Table 1. Portfolio components as evidence of teaching competence 


Competency 

Components reflecting the competency 

Planning the teaching and learning 
process 

Teaching philosophy 

Course and class planning documents 

Conducting the teaching process 

Self-analysis inventory 4 , class observation analysis 

Valuing the impact of the teaching and 
learning process 

Techniques and instruments used to assess students 
Self-assessment of student achievment (emphasizing 
evidence) and faculty evaluation results. 

Note: Elaborated by authors. 


4 Self-analysis inventory. It consists in obtaining a self-assessment of our own teaching abilities, in order to 
detect needs and interests for professional development. The instrument was a translation of the one 
developed by Janice Orrell (2001), based on the seven principles of Arthur W. Chickering and Zelda 
F. Gamson (1997). It consists of two sections: personal data (gender, age, level of education, type of work, 
work schedule, courses taught) and self-assessment. This section has 52 statements related to: instructor- 
student interaction, student-student interaction, active learning, timely feedback, time on task, high 
expectations, and respect to student diversity, talents and ways of learning, professional conduct. 
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Each component of the portfolio was assessed using rubrics developed by the authors of 
the model. 


3 . Findings 

The analysis of the data resulted in quantitative findings from each of the rubrics, 
findings that were then reviewed by the instructor, the other faculty members 
participating in the workshop, and the instructional specialists conducting the 
workshop. 

The faculty members reported the numerical results not to be a useful part of 
developing the portfolios. Instead, the instructors stated that the process of preparing 
the portfolios made them aware of inconsistencies between their teaching philosophy 
and their practice. Particularly they realized almost no inconsistencies between their 
course planning/ assessment with the curriculum plan, partly because both were based 
on a competency model. However, they began to discuss the advantages and 
disadvantages of using this formative evaluation approach (Bloom, Hastings & Madaus, 
1971 ; Scriven, 1967 ). 

In analyzing the video tapes of their teaching, the instructors realized the need for 
improving their practice in relation to expectations about their students and the need for 
providing them with feedback in timely matter. In addition, instructors identified 
important aspects they needed to improve interaction with their students. Much of their 
analysis focused on lecturing as the main method of teaching. 

According to the instructors, the main value of developing the portfolios was its 
potential for reflection and dialogue. Professors not only were able to reflect on their 
strengths, weaknesses, and areas for improvement, they also began to engage in 
reflection as a community of practice (Johnson-Lenz & Johnson-Lenz, 1997 ). After the 
workshop, the instructors began to share their classroom experiences, and began to use 
the portfolios to initiate discussion about how to improve teaching in their college. They 
looked for what they could do to help each other improve teaching. The analysis 
changed from individual to group, a new dialogue started about how the teaching of an 
individual professor related to the others teaching the curriculum. 

There are two functions here, portfolio development and portfolio reflection, not 
necessarily sequential. For some purposes, it is important to complete the portfolio, to 
have a product for reflection and display. For formative purposes, the act of 
contemplating and designing the portfolio provides simultaneous opportunity to 
diagnose teaching behavior. It is a form of needs assessment. Contemplated portfolios 
can be an instrument for the improvement of teaching. 


4. Conclusions 

This paper described the preliminary findings of a study using portfolios for improving 
college teaching at a university in Southern Mexico. The scrutiny took place during a 
workshop that allowed instructors to initiate a dialogue about teaching in the context of 
their institution, their teaching philosophy as well as an analysis of their practice and 
their peers’ practice, 

Even though the portfolios did not provide an ethnographic analysis of the instructors’ 
teaching, their conceptualization allowed the instructors to collect evidence of the 
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quality of their work. The workshop provided a dialogue for analysis of their classroom 
practice. This is important augmentation to a single instrument or a single source of 
information. Multiple measures can lead to better understanding of what is needed for 
improving of teaching quality (Gardner, 1983; Stake & Cisneros-Cohernour, 2004). 

Using portfolios in a workshop contributed to analysis and dialogue among the 
participants and the nurturing of a community of practice. The workshop participants 
became “a group of professionals informally united, . . . exposed to common problems 
and looking for common solutions, [(becoming themselves]) a source of knowledge 
(Johnson-Lenz & Johnson-Lenz, 1998; Stake & Cisneros-Cohernour, 1999). 

The creation of communities of practice among teachers can be vital not only for 
assessment purposes, but for developing and improving teaching quality. As a wide 
range of research indicates, one of the best formats for teacher to learn and improve is 
when they work together with others. Formative evaluation work comfortably involves 
working together in these communities to develop a qualitative appraisal of teaching. 
The administrator’s responsibility is to encourage the work of the community and to 
facilitate the conditions needed for change. Setting a goal of portfolio development can 
be useful, where the act of contemplation and design provides a form of needs 
assessment. The portfolio itself may be the lesser goal. Reflexive contemplation may be 
the greater. 
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